Performance Comparison of Learning to Rank Algorithms for Information Retrieval

Learning to rank is the problem of ranking objects by using machine learning techniques. One of the applications of learning to rank is for ranking document of search results. In this research, we compare the performance of three learning to rank algorithms: RankSVM, LambdaMART, and Additive Groves. RankSVM, which is ranking variant of the classical SVM algorithm, is commonly used as a baseline in learning to rank experiments. LambdaMART and Additive Groves is both tree ensembles algorithm. They belong to the class of algorithms that yield top results in the recent Yahoo! Learning to Rank Challenge. The comparison is performed by evaluating the results algorithms to a standard dataset. We also study the outcome of performing feature selection for final algorithm performance. Two feature selection techniques based on the filter and wrapper approaches are implemented. The experiment is conducted with the LETOR 4.0 dataset. This dataset consists of 46 features, such as: TF, BM25, LMIR, PageRank scores, etc. Parameter tuning is performed beforehand in order to find the best parameters for reliable comparison. The evaluation metric is Normalized Discounted Cumulative Gain (NDCG), a graded relevance measure for ranking results. The experimental results reveal that Learning to Rank algorithms outperform the conventional algorithms. In addition, applying feature selections improves the algorithm performances. Keywords—AAAZ learning to rank algorithms; RankSVM; Lambda MART; Additive Group; Feature Selection.